UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. : 6,914,599 Bl Page 1 of 8 

DATED : July 5, 2005 

INVENTORY) : Simon Michael Rowe et al. 



It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Title page. 

Item [56], References Cited, OTHER PUBLICATIONS, "Haralick et al, 
"Computer and Robot Vision Volume 2", 1993.*" should be deleted. 
Item [57], ABSTRACT, 

Line 17, "discontinuous" should read — discontinuities 
Drawings. 

Sheet 3 should be replaced by attached sheets 3 of 19 and 4 of 19. 
Sheet 1 1 should be replaced by attached sheets 12 of 19 and 13 of 19. 
Sheet 12 should be replaced by attached sheets 14 of 19 and 15 of 19. 

Column 2. 

Line 53, "FIG. 3" should read -- FIG. 3, 
Column 5. 

Line 54, "FIG. 10 image" should read - image --. 
Column 19. 

Line 54, "for-the" should read — for the --. 
Column 22. 

Line 13, "def ine" should read ~ define --; and 
Line 55, "object" should read - objects --. 

Column 32. 

Line 65, "for" should read ~ means for --. 



UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. : 6,914,599 Bl Page 2 of 8 

DATED : July 5, 2005 

INVENTOR(S) : Simon Michael Rowe et al. 



It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Column 56. 

Line 40, "become" should read ~ to become --. 



Signed and Sealed this 
Twenty-third Day of May, 2006 




JONW. DUD AS 
Director of the United States Patent and Trademark Office 



Page 3 of 8 



U.S. Patent 



Jul. 5, 2005 



Sheet 3 of 19 6,914,599 Bl 



SET UP 3D MODEL OF STATIC 
BACKGROUND 



S2 



SET BACKGROUND IMAGE 
PARAMETERS FOR FIRST 
CAMERA 



RECORD IMAGES OF *ACTION" 
WITH FIRST CAMERA 



S3a 



S4a 



SET BACKGROUND IMAGE 
PARAMETERS FOR SECOND 
CAMERA 



RECORD IMAGES OF "ACTION" 
WITH SECOND CAMERA 



EXTRACT FOREGROUND 
PORTIONS OF NEXT IMAGE 
FROM FIRST CAMERA AND 
PROJECT INTO 3D WORLD 
SPACE 



S6a 



S3b 



S4b 



EXTRACT FOREGROUND 

PORTIONS OF 
CORRESPONDING IMAGE 
FROM SECOND CAMERA AND 
PROJECT INTO 3D WORLD 
SPACE 



S6b 



PROCESS SHADOWS 



PROCESS TO PREVENT 
SEPARATE OBJECTS 
APPEARING AS COMPOSITES 



S8 



S10 



MODEL OBJECTS IN 3D 
WORLD SPACE FOR FIRST 
CAMERA 



STORE TIME-STAMPED 3D 
OBJECT DATA FOR FIRST 
CAMERA 



® 



•SI 1a 



S12a 



I 



MODEL OBJECTS IN SO 
WORLD SPACE FOR 
SECOND CAMERA 



STORE TIME-STAMPED 30 
OBJECT DATA FOR SECOND 
CAMERA 



•S11b 



•S12b 



FIG. 3a 



Page 4 of 8 



U.S. Patent jui. 5, 2005 sheet 4 of 19 6,914,599 Bl 



® 



YES 




YES 



S16 



FIG. 3b 



i 



RECORD REFERENCE J MAGES OF 
BACKGROUND 



/ 



S22 



CALCULATE IMAGE SPACE TO 30 
WORLD SPACE TRANSFORMATION 
FOR UNKNOWN CAMERA POSITION 
ANO PARAMETERS 



/ 



S24 



CALCULATE BACKGROUND IMAGE 
PIXEL PARAMETERS 



T 



S26 



FIG. 4 



Page 5 of 8 



U.S. Patent 



NO 



Jul. 5, 2005 



Sheet 12 of 19 6,914,599 Bl 



DEFINE VIEWING PARAMETERS 



V 



$296 



is 



$296 



SELECTED VIEWING DIRECTION 
^WITHIN PREDETERMINED ANGLE OF. 
.VERTICAL?- 
r YES 



RENDER DATA SCHEMATICALLY REPRESENTING 
OBJECT POSITIONS ON GROUND PLANE TO 
FRAME BUFFER 



S300 



SELECT OBJECT DATA 



V 



RENDER TO FRAME BUFFER 



V 



S302 



S304 




RENDER OBJECT DATA FOR OTHER CAMERA 
TO FRAME BUFFER 



$308 



COMPARE IMAGES 



V 



S310 




COMBINE IMAGE DATA TO SMOOTH TRANSITION 



S314 



® © 



FIG. 14a 



Page 6 of 8 



U.S. Patent jui. 5, 2005 sheet 13 of 19 6,914,599 Bl 



I 



DETERMINE IMAGE QUALITY INFORMATION 



V 



2-D VIDEO IMAGE 



V 



S316 



S318 




YES 



FIG. 14b 



Page 7 of 8 



U.S. Patent 



Jul. 5, 2005 Sheet 14 of 19 6,914,599 Bl 



COMPARE VI EW DIRECTION WITH CAMERA AXES AND 
IDENTIFY CAMERA WITH AXIS CLOSEST TO VIEW OIRECTION 



S400 



S402 



IS AXIS 

OF NON-IDENTIFIED CAMERA WITHIN 
± PREDETERMINED ANGLE OF VIEW 
DIRECTION? 



COMPARE CAMERA IMAGE DATA TRANSFER METHODS 



S404 




SELECT OBJECT DATA FROM CAMERA WITH HIGHEST ]/ 
QUALITY TRANSFER METHOD 



a. 



COMPARE CAMERA RESOLUTIONS 



V 



S408 



S410 




SELECT OBJECT DATA FROM CAMERA WITH HIGHEST 
RESOLUTION 



-S414 



3l 



COMPARE IMAGE STABILITIES 



V 



S416 




SELECT OBJECT DATA FROM CAMERA WITH HIGHEST I/' 
STABILITY 



S420 



FIG. 15a 



® 



Page 8 of 8 



U.S. Patent jui. 5, 2005 



Sheet 15 of 19 6,914,599 Bl 



I 



COMPARE NUMBER OF OCCLUDED OBJECTS FOR EACH 
CAMERA 



S422 



YES 




SELECT OBJECT OATA FROM CAMERA WHICH 
PROOUCES SMALLEST NUMBER OF OCCLUSIONS 



S426 



SELECT 08JECT DATA FROM CAMERA WITH VIEWING 
DIRECTION CLOSEST TO VIEWING DIRECTION CHOSEN 
BY USER 



T 



/ 



S428 



FIG. 15b 



